Accuracy and reliability of retinal photo grading for diabetic retinopathy: Remote graders from a developing country and standard retinal photo grader in Australia

BACKGROUND To evaluate the accuracy and reliability of fundus retinal photos graded by local graders in Bangladesh with those graded by an expert at the Centre for Eye Research Australia (CERA) in the context of mass scale diabetic retinopathy (DR) screening in Bangladesh. METHODS A population-based cross-sectional study of 3,104 adults identified 213 (7.2%) eligible patients with diabetes of age ≥ 40 years in 2012-2013. Retinal photographs were collected using a non-mydriatic digital fundus retinal camera and a two-field imaging protocol. The photos were graded by two remote graders (G1 and G2) who were trained by a retinal specialist (RS) in Bangladesh, by the RS himself, and by a Centre for Eye Research Australia (CERA) grader. The local graders up skilled their grading ability by comparing 30% of the photos graded by the CERA grader with their own grades. Learning from that exercise was applied to the remaining 70% of photos, which were re-graded. Reliability and accuracy of grading amongst the graders were reported using cross tabulation, inter- and intra-grader reliability, and with sensitivity and specificity. RESULTS Of 122 eyes from 61 patients, the mild (R1) DR was estimated to be 14 to 25%, pre-proliferative (R2) DR 4-8%, and proliferative (R3) DR 0.8 to 1.6%, whereas 25%, 8%, 18%, and 15% were found to be ungradable by CERA, RS, G1, and G2, respectively. Of 8 (6.6%) eyes identified as R2 by the CERA grader, 5 (63%), 3 (38%) and 3 (38%) were correctly classified as R2, whereas the rest were classified either as R1 or R3 but none were classified as no DR (R0) or ungradable by the RS, G1 and G2, respectively. After getting experience reviewing the 30% test set graded by the CERA grader, the local graders graded moderate and severe DR with 100% accuracy. After excluding ungradable photos, the sensitivity (specificity) relative to the CERA grader was 82% (88%) before and 80% (93%) after training for G1 and 56% (87%) before and 77% (90%) after training for G2. In case of maculopathy, the CERA grader reported 11.2% eyes with maculopathy, which included 100% of the 4.9% by RS, 6.6% by G1, and 7.4% by G2. CONCLUSIONS Local graders in Bangladesh are able to grade retinal photos with high accuracy if the DR is at least of a moderate level. With appropriate training and experience, local graders have the ability to contribute significantly to the grading of millions of retinal photos, which required grading in resource- poor countries.